Trends in big data analytics

One of themajor applications of future generation parallel and distributed systems is in big-data analytics. Data repositories for such applications currently exceed exabytes and are rapidly increasing in size. Beyond their sheermagnitude, these datasets and associated applications’ considerations pose significant challenges formethod and software development. Datasets are oftendistributed and their size andprivacy considerations warrant distributed techniques. Data often resides on platforms with widely varying computational and network capabilities. Considerations of fault-tolerance, security, and access control are critical inmany applications (Dean and Ghemawat, 2004; Apache hadoop). Analysis tasks often have hard deadlines, and data quality is a major concern in yet other applications. For most emerging applications, data-driven models and methods, capable of operating at scale, are as-yet unknown. Even when known methods can be scaled, validation of results is amajor issue. Characteristics of hardware platforms and the software stack fundamentally impact data analytics. In this article, we provide an overview of the stateof-the-art and focus on emerging trends to highlight the hardware, software, and application landscape of big-data analytics. © 2014 Elsevier Inc. All rights reserved.